Inigo dev #118
Merged
Inigo dev #118
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
inigoval
commented
May 23, 2024
inigoval
commented
- Add timm ViT fine-tuning support
- Add sklearn linear regression sanity check to classification finetuning (TODO: add to regression also)
- Some formatting also
mwalmsley
reviewed
May 24, 2024
zoobot/pytorch/training/finetune.py
Outdated
| self.encoder = encoder | ||
| # work out encoder dim 'manually' | ||
| self.encoder_dim = define_model.get_encoder_dim(self.encoder) | ||
| if isinstance(self.encoder, timm.models.VisionTransformer): |
There was a problem hiding this comment.
TODO for me, I later found out this actually works on all timm models so should add a check like this for everything
mwalmsley
reviewed
May 24, 2024
zoobot/pytorch/training/finetune.py
Outdated
|
|
||
| # Sanity check embeddings with linear evaluation first | ||
| def on_train_start(self) -> None: | ||
| with torch.no_grad(): |
There was a problem hiding this comment.
This makes sense for you and I but I'm not sure it should run by default for Zoobot. I think it may confuse astronomers who will now get two separate performance measurements for linear eval. I'm also a bit worried about the potential memory use from encoding everything. Maybe we can just wrap this in a new arg to ZoobotAbstract, linear_sanity_check=False, and an if statement/func here?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.